There are large differences in shape, texture and color of images in remote sensing image datasets, and the classification accuracy of remote sensing scenes is low due to the scale differences cased by different shooting heights and angles. Therefore, a Feature Aggregation Compensation Convolution Neural Network (FAC-CNN) was proposed, which used active rotation aggregation to fuse features of different scales and improved the complementarity between bottom features and top features through bidirectional gated method. In the network, the image pyramid was used to generate images of different scales and input them into the branch network to extract multi-scale features, and the active rotation aggregation method was proposed to fuse features of different scales, so that the fused features have directional information, which improved the generalization ability of the model to different scale inputs and different rotation inputs, and improved the classification accuracy of the model. On NorthWestern Polytechnical University REmote Sensing Image Scene Classification (NWPU-RESISC) dataset, the accuracy of FAC-CNN was increased by 2.05 percentage points and 2.69 percentage points respectively compared to those of Attention Recurrent Convolutional Network based on VGGNet (ARCNet-VGGNet) and Gated Bidirectional Network (GBNet); and on Aerial Image Dataset (AID), the accuracy of FAC-CNN was increased by 3.24 percentage points and 0.86 percentage points respectively compared to those of the two comparison networks. Experimental results show that FAC-CNN can effectively solve the problems in remote sensing image datasets and improve the accuracy of remote sensing scene classification.